Gustavo Díaz
McMaster University
diazg2@mcmaster.ca
@gusvalo
Verónica Pérez Bentancur
Universidad de la República veronica.perez@cienciassociales.edu.uy
@veroperezben
Ines Fynn
Pontificia Universidad Católica de Chile
ifynn@uc.cl
@ifynn_
Lucía Tiscornia
University College Dublin
lucia.tiscornia@ucd.ie
@tiscornia21
Slides: gustavodiaz.org/talk
Social scientists care about sensitive issues
Asking about them directly leads to misreporting
Solution: Indirect questioning techniques
List experiments popular in political science
Now I am going to read you things that make people angry or upset
After I read them all, tell me HOW MANY of them upset you
I don’t want to know which ones, just tell me HOW MANY
I don’t want to know which ones, just tell me HOW MANY
I don’t want to know which ones, just tell me HOW MANY
Negatively correlated items (Glynn 2013)
Covariate adjustment (Blair and Imai 2012)
Auxiliary information (Chou 2020)
Double list experiments (Diaz 2023)
Combine with direct questions (Aronow et al 2015)
Negatively correlated items (Glynn 2013)
Covariate adjustment (Blair and Imai 2012)
Auxiliary information (Chou 2020)
Double list experiments (Diaz 2023)
Combine with direct questions (Aronow et al 2015)
\[ \hat{\mu} = \overline{Y} + (1 - \overline{Y}) (\overline{V}_{1,0} - \overline{V}_{0,0}) \]
\(\overline{Y}\): Proportion confess in direct question
\((\overline{V}_{1,0} - \overline{V}_{0,0})\): List experiment estimate among those not confessing
Can’t always include direct questions
Need an indirect questioning technique that lets us infer individual responses to sensitive item
But most rely on anonymity
Can’t combine without extra assumptions or altered designs (e.g. Blair, Imai, and Lyall 2014)
How many people do you know,
How many people do you know, who also know you,
How many people do you know, who also know you, with whom you have interacted in the last year
How many people do you know, who also know you, with whom you have interacted in the last year in person, by phone, or any other channel.
Assumption
If someone knows a unusually large number of people with the sensitive item, then they are likely to hold the sensitive item too.
\[ \begin{align*} y_{ik} \sim \text{negative-binomial}( & \text{mean} = e^{\alpha_i + \beta_k},\\ & \text{overdispersion} = \omega_k) \end{align*} \]
\(y_{ik}\): Degree of group \(k\) for person \(i\)
\(\alpha_i\): Expected degree of person \(i\) (logged)
\(\beta_k\): Expected degrees of group \(k\) (logged)
\(\omega_k\): Controls variance in propensity to know someone from group \(k\)
Fit with MLE in two steps (Personal network, sensitive group network)
Focus on standardized residuals:
\[ r_{ik} = \sqrt{y_{ik}} - \sqrt{e \alpha_i + \beta_k} \]
Low crime, but embedded
Even here criminal organizations replace government
Fieldwork: Interactions are sensitive topic
Goal: Document extent of exposure to criminal governance strategies (positive, negative)
Facebook sample of Montevideo residents (N = 2688)
Four criminal governance strategies
Facebook sample of Montevideo residents (N = 2688)
Four criminal governance strategies
During the last six months, in your neighborhood, have you seen gangs…
Things people have experienced in the last six months:
| List A | List B |
|---|---|
| Saw people doing sports | Saw people playing soccer |
| Visited friends | Chatted with friends |
| Activities by feminist groups | Activities by LGBTQ groups |
| Went to church | Went to charity events |
Things people have experienced in the last six months:
| List A | List B |
|---|---|
| Saw people doing sports | Saw people playing soccer |
| Visited friends | Chatted with friends |
| Activities by feminist groups | Activities by LGBTQ groups |
| Went to church | Went to charity events |
| Did not drink mate | Gangs threatening neighbors |
Things people have experienced in the last six months:
| List A | List B |
|---|---|
| Saw people doing sports | Saw people playing soccer |
| Visited friends | Chatted with friends |
| Activities by feminist groups | Activities by LGBTQ groups |
| Went to church | Went to charity events |
| Gangs threatening neighbors | Did not drink mate |
How many people do you know, who also know you, with whom you have interacted in the last year in person, by phone, or any other channel
15 reference groups + sensitive item
Choice range 0-10+
Recode as 1 if \(r_{ik} > \text{mean}(r_{ik}) + 1 \text{SD}\), 0 otherwise
| Estimator | List A | List B |
|---|---|---|
| List | 0.086 | 0.088 |
| List + direct | 0.078 | 0.080 |
| List + NSUM | 0.073 | 0.078 |
Can use NSUM to improve precision of list experiment estimates
Logic applies to other indirect questioning techniques that rely on anonymity
More principled approach to infer who holds the sensitive item from NSUM questions?
What would you like to see to be convinced that NSUM is a suitable replacement for direct questions?
From Las Piedras
Male 25-29
Police officers
University students
Had a kid last year
Passed away last year
Married last year
Female 45-49
Public employees
Welfare card holders
Registered with party
With kids in public school
Did not vote in last election
Currently in jail
Recently unemployed [Sensitive item]